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How can we eliminate Human Error? 


Tadmexelereilela 


¢ Human Error - “An action which fails to produce the expected results 
and therefore, leads to an unwanted consequence” (Hollnagel, 1993). 


¢ Person approach 
¢ Systems approach 


“The pursuit of greater safety is seriously impeded by an approach that 
does not seek out and remove the error proving properties within the 
system at large” (Reason, 2000). 
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Problem Statement 


Research results have shown that more than half of aviation, 
aerospace and aeronautics mishaps/incidents are attributed to 
human error. As a part of Quality within space exploration ground 
processing operations, the identification and/or classification of 
underlying contributors and causes of human error must be 
identified, in order to manage human error. 

Most incident reports are not designed around a theoretical 
framework of human error (Wiegmann and Shappell, 2001). 
Report systems have been beneficial for identifying engineering and 
mechanical failures, but often fail to address the core issues and 
causes of the failure due to human error (Wiegmann and Shappell, 
2001). 

This makes the intervention and integration of a strategy to reduce 
the human error become difficult (Wiegmann and Shappell, 2001). 
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D & OPERATIONS 


¢ Dynamics of the Generic Error Model System (GEMS) used for 
relating Reasons three basic error types to Rasmussen's three 
performances levels (Reason, 1990). 


Performance Description 
Error Type 
Level 


Automated non-cognizant errors of automatic 
processing (attention/memory) during regular 
routine actions that are identified quickly 
(Reason, 1990). 
Errors of rule-based behavior. For example: 
Rule-based applying the wrong rule for a give situation (often 
mistakes with a tendency to keep repeating the same wrong 
actions “strong but wrong’). 
Errors of cognitive (knowledge-based) processing 
whereby a problem is not analyzed correctly (or not 
at all) and this results in an error (e.g. wrong 
response to a multitude of alarms based on an 
incomplete understanding of the actual problem). 


Skill-based level | Slips and lapses 


Rule-based level 


Knowledge-based | Knowledge-based 
mistakes 
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esp Theoretical Framework 


GROUND SYSTEMS 


DEVELOPMENT OPERATIONS 


GEMS is an effort to provide an integrated framework of the error types operating at all three 
levels of performance: Skill based, Rule based and Knowledge based. This is a hybrid of two 


sets of error theories proposed by Norman (1981) and Reason and Mycielska (1982) (Reason, 
1987). 


Basic Error 


Type 


La Based 
La 


Attentional Failures 


Unintended 
Action 


Intrusion, Misordering, Omission, 
Reversal, Mistiming 
onsite Failures 


Omissions, onsite reduced 
intentionality 


Rule based mistakes 


Misapplication of good rule, 
Application of bad rule 


Knowledge based mistakes 


Confirmation bias, Selectivity, 
Vagabonding 


Intended Action 


Routine violation 
Exceptional violation 
Acts of Sabotage 


Figure 1: Generic Error Modeling System — GEMS (Reason, 1990) 


National Aeronautics and Space Administration 


Research Objective 


OPERATIONS 


¢ Provide a framework and methodology using the Human Error Assessment 
and Reduction Technique (HEART) and Human Factor Analysis and 
Classification System (HFACS), as an analysis tool to identify contributing 
factors, their impact on human error events, and predict the Human Error 
probabilities (HEPs) of future occurrences. 

¢ This research methodology was applied (retrospectively) to six (6) NASA 
ground processing operations scenarios and thirty (30) years of Launch 
Vehicle related mishap data. This modifiable framework can be used and 
followed by other space and similar complex operations. 


Some holes due to active failures, 
others due to latent conditions 


Hazards 


Cheese Model, Successive 
Layers of Defenses 


Losses 
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Research Question and Hypotheses 


Research Question 


¢ What are the identified leading human error causes and contributors to 
historical Launch Vehicle Ground Processing Operations mishaps and 
findings based on past mishaps, near mishaps, and close calls? 

Hypothesis 1 

¢ Contributing factors: unsafe acts of operators, preconditions for unsafe 
acts, unsafe supervision, and/or organizational influences (multiple 
causes) do not have an impact on human error events (i.e. mishaps, 
close calls, incident or accidents) in NASA ground processing 
operations. 


Hypothesis 2 


¢ The HFACS framework conceptual model can be proven to be a viable 
analysis and classification system to help classify latent and active 
failures, underlying contributors and causes of human error in NASA 
ground processing operations. 
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~) Research Hypotheses continued 


& OPERATIONS 


Hypothesis 3 


¢ The development of a model using the HEART assessment can be used as a 
tool to help determine the probability of human error occurrence, in order to 
help minimize human error in NASA ground processing operations. 

¢ Independent variable : Contributing Factors (identified by the SMEs for 
specific Scenarios of tasks performed for NASA KSC Ground Processing 
Operations). 


¢ Dependent Variable: Probability of a Human error event (i.e. mishaps, close 
calls, incident or accidents). 
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Preliminary Analysis 


¢ Conducted informal Subject Matter Expert (SME) discussions (8 SMEs with 34, 
31 and 30 years KSC GPO experience). 

¢ HEART Generic Tasks - SMEs identified and categorized examples of historical 
shuttle specific ground processing operations. 

¢ HEART Surveys - SMEs identified general tasks and error producing conditions 
associated with three (8) NASA KSC Ground Processing Area Scenarios used 
in research. 

¢ HFACS — SMEs modified HFACS’ four (4) Levels of Human Error examples with 
the specific KSC examples for categorization. 
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ENT & OPERATIONS 


HFACS Sublevels/HEART Error 
Producing Conditions 
HFACS 4 Levels of Human Error 


GPO Tasks 
(e.g. HEART 
Generic Tasks) 


Unsafe Acts 


Preconditions of 


Scenarios/Locations 
Unsafe Acts 


Unsafe Supervision 


Organizational 
Influences 


NASA KSC Ground Processing Operations Human Error Framework (Alexander, 2016) 
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esip) Methodology Overview 


OPERATIONS 


Literature 
Review 
Existing Gap 


ia 
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Results - Human Factors Analysis and Classification 


System (HFACS) 


¢ Mishap data from October 1984 — May 2014 was categorized and 
sorted into HFACS Four Levels of Human Error categories. 


HFACS Overview of Error Type Results Definition/Examples (Shappell, 2012) 
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GSDO@ Results - Human Factors Analysis and Classification 


COUNTED Ove ete System (HFACS) 


¢ Data categorized into 8 HFACS Sublevels: Skilled Based, Routine 
violations, Crew Resource Management, Perceptual Errors, 
Exceptional Violations, Decision Based, Physical Environment and 
Supervisory Violation. 


Crew Resource Physical = HFACS Human Error Factors Percentage 
Management Environment 


4% Supervisory 


Violation 
0.005% 


Decision Based 3% 


HFACS KSC GPO Frequency Counts 
(Alexander, 2016) 


Skilled Based 
46% 


Routine Violation 


Exceptional 
Perceptual _/ Violation 
4% 3% 


1. 
1 


= Skilled Based (Level 1) = Routine Violations (Level 1) 


= Crew Resource Management (Level 2) = Perceptual Errors (Level 1) 
m= Exceptional (Level 1) = Decision Based (Level 1) 


= Physical Environment (Level 2) = Supervisory Violation (Level 3) 
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Sse Results - HEACS 
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DEVELOPMENT & OPERATIONS 


Binary Logistics Regression Analysis 
— Occurrence of a Mishap identified as “response/event” and 
represented by and “0” or “1”. 
¢ The Regression Analysis generated a regression model in which the 
predicted probability of each occurrence was calculated. 
¢ Binary Logistic Regression Expression for this Model is: 
P(1j)=eM/(1+e™)) 


HFACS Human Error Fitted 
r Probability 


Facto 
Skilled Based 
Perceptual Errors 


Routine Violation 
Exceptional Violation 


Crew Resource 
Management 
Physical Environment 


Supervisory Violation 
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SsSsDG Results - HEACS 


DEVELOPMENT & OPERATIONS 


Binary Logistics Regression Analysis with Stepwise Backward Elimination 
¢ Binary Logistic Regression Expression for this Model is: 
P(1j)=e™/(14+e™)) 


HFACS Human Error Fitted P Odds 
value ate lite) 


27% 0.000 3.54 
41% 0.005 6.60 
52% 0.000 10.33 
36% 0.013 5.38 
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3DG Results -HEART Survey Participants 


ERATION 


Human Error Assessment and Reduction Technique (HEART) 

¢ Participation Goal: At least 18 subjects for all Scenario assessments based on 
Power Curve test with a Variance: 0.707, Std Dev: 0.5, Power: 0.80, a: 0.05 

¢ Total Participants: 41 

¢ Participants’ average years working at KSC: 23.9875 years (Range 10 to 37 
years). 

¢ Participants’ average years working at KSC supporting Ground Processing 
Operations: 19.8125 years (Range 3 to 33 years). 


Power Curve for Paired t Test 


s 
\ ee 
Pate 


0.0 0.2 
Difference 
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GSDO@ Results - HEART Survey Participants 


NERS i oles 


GF aN OU MENT & OPERATIONS 


¢ Subjects were asked to answer a total of 21 questions. 
3 job related questions for statistical purposes 
— 18 evaluation questions involving 3 NASA KSC GPO Scenarios (VAB, 
OPFs and Pads A/B). 
— Assessment based on the level of affect an Error Producing Condition 
(EPC) may have on a specific GPO task (very low, low, moderate, high, 


and very high). 
SURVEY PARTICIPANTS' JOB FUNCTION 
__ Unidentified 
Human Factors ee 2% 
Modeling, QAS____ 7 
and Safety 
Specialist 
2% 
Technican_——_| 
27% \_ Quality Assurance 
Specialist (QAS) 
42% 
Quality 
Engineer/Systems 
Engineer 
2% 
QAS and Safety_~ __ Inspector (Quality 
Specialist Control) 
15% 10% 
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GsDG Results - HEART Human Error Probability 


1 = as =i a OPERATIONS 


¢ Of the calculated Human Error Probability (HEP) values, three (3) values had 
the highest probability (22%, 22% and 26%). 


¢ The three highest HEP values were from survey questions 5, 19, and 20, 
which all had an EPC commonality. 


Accessibility Limitations, Physical Stress and Tiredness. 
— Poor lighting deals with the Physical environment. 


Generic 
Generic Task, 
Task, HEART 
HEART Nominal 
Nominal Human 
Assessed Human Error Prob. 
Proportion of Assessed Assessed Assessed Error Prob. Value 
Survey Affect Factor — (HEPs) (HEPs) 
Question a (average) (Table 20) (Table 20) 


2 A ZA 
Pp .495 | 1.396 | c ° 


lo) 


pT 617% | 


m 


Las 
i 
1 ee 


0.603125 124125} Od re | ee | ooos 
0.529 vaio 0.008 


m jm 


ny) 


na 


4] 
2 ae 
ee 
a 


omar 0.552439 2.657317 1 —— il — ene? 


0.47622 1380976 | 
= 0.448205 1.358564 | __1.179282 — a aoe 
P| 0.573415 tos7aqa2T Tt 17% J 
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DEVELOPMENT & OPERATIONS 


Binary Logistic Regression model with Stepwise Backward Elimination 
¢ All factors’ p values are < 0.05, thus statistically significant. 


¢ All three Goodness-to-Fit tests’ p values were greater than 0.05, 
indicating we want to reject the first null hypotheses (H,). 


HFACS Human Error Odds 
Factor Ratio 
Perceptual Error 0.000 10°33 


Exceptional Violation 0.013 


Goodness-of-Fit tests 
Skilled Based 0.000 


Deviance 
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O@ Model Validation - HFACS 


OPERATIONS 


¢ The HFACS model was verified by consistency and comparison to other 
research conducted with the HFACS Classification system and data in the 
aeronautics field. 


¢ Remotely Piloted Aircraft (RPA) Operations (Tvaryanas, 2006) 


— Prior HEFACS RPA mishap studies, 3 of the 5 studies’ largest percentage 
of mishaps fell under the “Unsafe Acts.” 


¢ Remotely Piloted Aircraft Mishaps HFACS analysis on recurrent error 
pathways on 95 Remotely Piloted Aircraft Mishaps (Tvaryanas, 2008). 


— “Perceptional” and “Skilled” Based Error pathways (under HFACS 
“Unsafe Acts”) and had common associated latent failures. 


— Together accountable for the majority of crewmember related mishaps. 
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UND SYSTEMS 


GF rn ¢ OU MENT OPERATIONS 


Model Validation - HFACS 


¢ Republic of China (ROC) Air Force study 


— Ten (10) highest ranking frequencies of occurrence fell within the 
“Unsafe Acts (Level 1)” and “Preconditions for unsafe acts (Level 


2)’ HFACS Categories. 


Republic of China (ROC) Air Force Study 
Top 10 Frequency Counts and Inter-rater Reliability 
(Li, W., 2006). 


@ Skilled-Based Errors (Level 1) B Decision Errors (Level 1) 

@ Adverse Mental State (Level 2) m Resource Management (Level 2) 
@ Inadequate Supervision (Level 3) m Violations (Level 1) 

@ Crew Resource Management (Level 2) m Perceptual Errors (Level 1) 


@ Organization Process (Level 4) @ Physical environment (Level 2) 
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HFACS KSC GPO Frequency Counts 
(Alexander, 2016) 


= Skilled Based (Level 1) = Routine Violations (Level 1) 
= Crew Resource Management (Level 2) = Perceptual Errors (Level 1) 
= Exceptional (Level 1) = Decision Based (Level 1) 
m Physical Environment (Level 2) m= Supervisory Violation (Level 3) 
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EVELOPMENT & OPERATIO 


Research Question 

¢ What are the identified leading human error causes and contributors to 
historical Launch Vehicle Ground Processing Operations mishaps and 
findings based on past mishaps, near mishaps, and close calls? Quantifying 
this data and identifying the leading cause will be essential in the research 
analysis. 

Results 

¢ The Binary Logistics Regression Equation with Stepwise Backward 
Elimination (simplified) Equation identified the significant causes and 
contributors as: 

Skilled Based Errors 

Decision Based Errors 

Perceptual Errors 

Exceptional Violations 
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Hypotheses Results 


COIs S VoulelsS 


OPERATIONS 


Hypothesis 1 

¢ Contributing factors : unsafe acts of operators, preconditions for 
unsafe acts, unsafe supervision, and/or organizational influences 
(multiple causes) do not have an impact on human error events (i.e. 
mishaps, close calls, incident or accidents) in KSC ground processing 
operations. 


Results 


¢ Binary Logistics Regression Model results support that mishaps 
categorized using the modified NASA KSC HFACS Model, show 
significant contributing factors to KSC Ground Processing Operations 
(GPO) Human Error. 


¢ Ability for KSC’s GPO related Mishaps to be sorted into the HFACS 
Levels and sub-categories support that the HFACS tool could be used 
for complex operations such as KSC GPOs. 
— Of the HFACS 4 Levels, the only level that did not have any KSC 
GPO mishaps was “Organizational Influences.” 


¢ Therefore, the H, null Hypothesis is REJECTED. Contributing HFACS 
factors DO have an impact on human error events. 
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Hypotheses Results 


Hypothesis 2 


The HFACS framework conceptual model can be proven to be a viable analysis 
and classification system to help classify both latent and active underlying 
contributors and causes of human error in KSC ground processing operations. 


Results 


The HFACS framework conceptual model used in this research revealed both 
active and latent failures. The majority of the significant contributing factors were 
from HFACS Levels 1 and 2, which encompass both active and latent failures. 


This research revealed human error underlying contributors and causes based on 
Ws HFACS framework and identified with the Binary Logistics Regression 
quations. 


In the Model Validation of this research, the HFACS model was verified by 
consistency and comparison to other research conducted with the HFACS 
Classification system and data in the aeronautics field. 


Therefore, the Hy null Hypothesis is ACCEPTED. The HFACS framework 
conceptual model CAN be proven to be a viable analysis and classification 
system. 
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Hypotheses Results 


Hypothesis 3 


The development of a model using the HEART assessment can be used as a tool to help 
determine the probability of human error occurrence, in order to help minimize human error 
in KSC ground processing operations. 


Results 


Comparing the 3 highest HEART HEP survey values (22% - 26%) related to physical 
limitations, there is a correlation with the HFACS Physical Environment Fitted Probability of 
33% data (before Stepwise Backward Elimination). 


All of the SME generated HEART survey scenarios, tasks, subtasks and EPCs all fell under 
the HFACS Level 2 Preconditions of Unsafe Acts Levels. 


After gathering survey data and calculating the HEART HEPs, the HEART tool was 
speared in determining the probability of human error occurrence from the generated 
cenarios. 


Therefore the H, null hypothesis is ACCEPTED due to consistency with related HFACS 
aerospace studies classifying the majority of human error occurrences falling within the 
HFACS Level 1 and 2 categories. The development of a model using the HEART 
assessment CAN be used as a tool to help determine the probability of human error 
occurrence. 
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Orel alelersrela 


¢ Based on the HFACS and HEART results and validation, the KSC Ground 
Processing Operations Framework is confirmed as a valid approach for human 
error analysis. 


¢ The framework its flexible in that it allows modification. 


¢ HEART’s “Generic Tasks” can be modified to specific tasks performed in the 
Operation. 


¢ This research can be modified and used for complex operations, such as other 
Space Operations and Space Programs on an International Level. 


¢ This research can be modified for use in recorded or documented Quality 


surveillance data over a period of time, as it relates to Operations and failure 
occurrences. 


¢ All of the error producing conditions in the final stage of the KSC GPO 
framework, cover contributing factors for both models. 
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¢ The following set of steps is a research Methodology approach 
that Space Operations and other complex organizations may 
use to modify and apply to their unique processes. 


Quantitative 
Study - HEART 


a | in| oe 


Research Methodology Contribution (Alexander, 2016) 
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